Automatic Evaluation Metric for Machine Translation that is Independent of Sentence Length

We propose new automatic evaluation metric to evaluate machine translation. Different from most similar metrics, our proposed metric does not depend heavily on sentence length. In most metrics based on f-measure comparisons of reference and candidate translations, the relative weight of each mismatched word in short sentences is larger than it in long sentences. Therefore, the evaluation score becomes disproportionally low in short sentences even when only one non-matching word exists. In our metric, the weight of each mismatched word is kept small even in short sentences. We designate our metric as Automatic Evaluation Metric that is Independent of Sentence Length (AILE). Experimental results indicate that AILE has the highest correlation with human judgments among some leading metrics.